Pandas Profiling Report

Dataset statistics

Number of variables	26
Number of observations	30000
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	6.0 MiB
Average record size in memory	208.0 B

Variable types

NUM	22
CAT	4

Warnings

`ID` is highly correlated with `df_index`	High correlation
`df_index` is highly correlated with `ID`	High correlation
`BILL_AMT2` is highly correlated with `BILL_AMT1` and 1 other fields	High correlation
`BILL_AMT1` is highly correlated with `BILL_AMT2`	High correlation
`BILL_AMT3` is highly correlated with `BILL_AMT2` and 1 other fields	High correlation
`BILL_AMT4` is highly correlated with `BILL_AMT3` and 2 other fields	High correlation
`BILL_AMT5` is highly correlated with `BILL_AMT4` and 1 other fields	High correlation
`BILL_AMT6` is highly correlated with `BILL_AMT4` and 1 other fields	High correlation
`PAY_AMT2` is highly skewed (γ1 = 30.45381745)	Skewed
`df_index` has unique values	Unique
`ID` has unique values	Unique
`PAY_0` has 14737 (49.1%) zeros	Zeros
`PAY_2` has 15730 (52.4%) zeros	Zeros
`PAY_3` has 15764 (52.5%) zeros	Zeros
`PAY_4` has 16455 (54.9%) zeros	Zeros
`PAY_5` has 16947 (56.5%) zeros	Zeros
`PAY_6` has 16286 (54.3%) zeros	Zeros
`BILL_AMT1` has 2008 (6.7%) zeros	Zeros
`BILL_AMT2` has 2506 (8.4%) zeros	Zeros
`BILL_AMT3` has 2870 (9.6%) zeros	Zeros
`BILL_AMT4` has 3195 (10.7%) zeros	Zeros
`BILL_AMT5` has 3506 (11.7%) zeros	Zeros
`BILL_AMT6` has 4020 (13.4%) zeros	Zeros
`PAY_AMT1` has 5249 (17.5%) zeros	Zeros
`PAY_AMT2` has 5396 (18.0%) zeros	Zeros
`PAY_AMT3` has 5968 (19.9%) zeros	Zeros
`PAY_AMT4` has 6408 (21.4%) zeros	Zeros
`PAY_AMT5` has 6703 (22.3%) zeros	Zeros
`PAY_AMT6` has 7173 (23.9%) zeros	Zeros

Reproduction

Analysis started	2020-12-04 05:12:56.035767
Analysis finished	2020-12-04 05:16:20.356268
Duration	3 minutes and 24.32 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

df_index
Real number (ℝ_≥0)

HIGH CORRELATION
UNIQUE

Distinct	30000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	15201.1399
Minimum	0
Maximum	30202
Zeros	1
Zeros (%)	< 0.1%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	1702.95
Q1	7702.75
median	15202.5
Q3	22702.25
95-th percentile	28702.05
Maximum	30202
Range	30202
Interquartile range (IQR)	14999.5

Descriptive statistics

Standard deviation	8662.753906
Coefficient of variation (CV)	0.5698752832
Kurtosis	-1.198695871
Mean	15201.1399
Median Absolute Deviation (MAD)	7500
Skewness	-0.0009415584061
Sum	456034197
Variance	75043305.23
Monotocity	Strictly increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2047	1	< 0.1%
23873	1	< 0.1%
29988	1	< 0.1%
25894	1	< 0.1%
27943	1	< 0.1%
5416	1	< 0.1%
7465	1	< 0.1%
1322	1	< 0.1%
3371	1	< 0.1%
13612	1	< 0.1%
Other values (29990)	29990	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	< 0.1%
1	1	< 0.1%
2	1	< 0.1%
3	1	< 0.1%
4	1	< 0.1%

Value	Count	Frequency (%)
30202	1	< 0.1%
30201	1	< 0.1%
30200	1	< 0.1%
30199	1	< 0.1%
30198	1	< 0.1%

ID
Real number (ℝ_≥0)

HIGH CORRELATION
UNIQUE

Distinct	30000
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	15000.5
Minimum	1
Maximum	30000
Zeros	0
Zeros (%)	0.0%
Memory size	234.4 KiB

Quantile statistics

Minimum	1
5-th percentile	1500.95
Q1	7500.75
median	15000.5
Q3	22500.25
95-th percentile	28500.05
Maximum	30000
Range	29999
Interquartile range (IQR)	14999.5

Descriptive statistics

Standard deviation	8660.398374
Coefficient of variation (CV)	0.5773406469
Kurtosis	-1.2
Mean	15000.5
Median Absolute Deviation (MAD)	7500
Skewness	0
Sum	450015000
Variance	75002500
Monotocity	Strictly increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2047	1	< 0.1%
1322	1	< 0.1%
15629	1	< 0.1%
9486	1	< 0.1%
11535	1	< 0.1%
21792	1	< 0.1%
23841	1	< 0.1%
17698	1	< 0.1%
19747	1	< 0.1%
29988	1	< 0.1%
Other values (29990)	29990	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1	< 0.1%
2	1	< 0.1%
3	1	< 0.1%
4	1	< 0.1%
5	1	< 0.1%

Value	Count	Frequency (%)
30000	1	< 0.1%
29999	1	< 0.1%
29998	1	< 0.1%
29997	1	< 0.1%
29996	1	< 0.1%

LIMIT_BAL
Real number (ℝ_≥0)

Distinct	81
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	167484.3227
Minimum	10000
Maximum	1000000
Zeros	0
Zeros (%)	0.0%
Memory size	234.4 KiB

Quantile statistics

Minimum	10000
5-th percentile	20000
Q1	50000
median	140000
Q3	240000
95-th percentile	430000
Maximum	1000000
Range	990000
Interquartile range (IQR)	190000

Descriptive statistics

Standard deviation	129747.6616
Coefficient of variation (CV)	0.7746854124
Kurtosis	0.5362628964
Mean	167484.3227
Median Absolute Deviation (MAD)	90000
Skewness	0.9928669605
Sum	5024529680
Variance	1.683445568e+10
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
50000	3365	11.2%
20000	1976	6.6%
30000	1610	5.4%
80000	1567	5.2%
200000	1528	5.1%
150000	1110	3.7%
100000	1048	3.5%
180000	995	3.3%
360000	881	2.9%
60000	825	2.8%
Other values (71)	15095	50.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
10000	493	1.6%
16000	2	< 0.1%
20000	1976	6.6%
30000	1610	5.4%
40000	230	0.8%

Value	Count	Frequency (%)
1000000	1	< 0.1%
800000	2	< 0.1%
780000	2	< 0.1%
760000	1	< 0.1%
750000	4	< 0.1%

SEX
Categorical

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	234.4 KiB

female	18112
male	11888

Frequencies
Length

Value	Count	Frequency (%)
female	18112	60.4%
male	11888	39.6%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	6
Median length	6
Mean length	5.207466667
Min length	4

EDUCATION
Categorical

Distinct	4
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	234.4 KiB

university	14030
graduate school	10585
high school	4917
other	468

Frequencies
Length

Value	Count	Frequency (%)
university	14030	46.8%
graduate school	10585	35.3%
high school	4917	16.4%
other	468	1.6%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	15
Median length	11
Mean length	11.85006667
Min length	5

MARRIAGE
Categorical

Distinct	4
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	234.4 KiB

2	15964
1	13659
3	323
0	54

Frequencies
Length

Value	Count	Frequency (%)
2	15964	53.2%
1	13659	45.5%
3	323	1.1%
0	54	0.2%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

AGE
Real number (ℝ_≥0)

Distinct	56
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	35.4855
Minimum	21
Maximum	79
Zeros	0
Zeros (%)	0.0%
Memory size	234.4 KiB

Quantile statistics

Minimum	21
5-th percentile	23
Q1	28
median	34
Q3	41
95-th percentile	53
Maximum	79
Range	58
Interquartile range (IQR)	13

Descriptive statistics

Standard deviation	9.217904068
Coefficient of variation (CV)	0.2597653709
Kurtosis	0.04430337824
Mean	35.4855
Median Absolute Deviation (MAD)	6
Skewness	0.7322458688
Sum	1064565
Variance	84.96975541
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
29	1605	5.3%
27	1477	4.9%
28	1409	4.7%
30	1395	4.7%
26	1256	4.2%
31	1217	4.1%
25	1186	4.0%
34	1162	3.9%
32	1158	3.9%
33	1146	3.8%
Other values (46)	16989	56.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
21	67	0.2%
22	560	1.9%
23	931	3.1%
24	1127	3.8%
25	1186	4.0%

Value	Count	Frequency (%)
79	1	< 0.1%
75	3	< 0.1%
74	1	< 0.1%
73	4	< 0.1%
72	3	< 0.1%

PAY_0
Real number (ℝ)

ZEROS

Distinct	11
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.0167
Minimum	-2
Maximum	8
Zeros	14737
Zeros (%)	49.1%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.123801528
Coefficient of variation (CV)	-67.29350467
Kurtosis	2.720715042
Mean	-0.0167
Median Absolute Deviation (MAD)	1
Skewness	0.7319749269
Sum	-501
Variance	1.262929874
Monotocity	Not monotonic

Histogram with fixed size bins (bins=11)

Value	Count	Frequency (%)
0	14737	49.1%
-1	5686	19.0%
1	3688	12.3%
-2	2759	9.2%
2	2667	8.9%
3	322	1.1%
4	76	0.3%
5	26	0.1%
8	19	0.1%
6	11	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	2759	9.2%
-1	5686	19.0%
0	14737	49.1%
1	3688	12.3%
2	2667	8.9%

Value	Count	Frequency (%)
8	19	0.1%
7	9	< 0.1%
6	11	< 0.1%
5	26	0.1%
4	76	0.3%

PAY_2
Real number (ℝ)

ZEROS

Distinct	11
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.1337666667
Minimum	-2
Maximum	8
Zeros	15730
Zeros (%)	52.4%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.197185973
Coefficient of variation (CV)	-8.949807922
Kurtosis	1.57041773
Mean	-0.1337666667
Median Absolute Deviation (MAD)	0
Skewness	0.7905650222
Sum	-4013
Variance	1.433254254
Monotocity	Not monotonic

Histogram with fixed size bins (bins=11)

Value	Count	Frequency (%)
0	15730	52.4%
-1	6050	20.2%
2	3927	13.1%
-2	3782	12.6%
3	326	1.1%
4	99	0.3%
1	28	0.1%
5	25	0.1%
7	20	0.1%
6	12	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	3782	12.6%
-1	6050	20.2%
0	15730	52.4%
1	28	0.1%
2	3927	13.1%

Value	Count	Frequency (%)
8	1	< 0.1%
7	20	0.1%
6	12	< 0.1%
5	25	0.1%
4	99	0.3%

PAY_3
Real number (ℝ)

ZEROS

Distinct	11
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.1662
Minimum	-2
Maximum	8
Zeros	15764
Zeros (%)	52.5%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.196867568
Coefficient of variation (CV)	-7.201369245
Kurtosis	2.084435875
Mean	-0.1662
Median Absolute Deviation (MAD)	0
Skewness	0.8406818269
Sum	-4986
Variance	1.432491976
Monotocity	Not monotonic

Histogram with fixed size bins (bins=11)

Value	Count	Frequency (%)
0	15764	52.5%
-1	5938	19.8%
-2	4085	13.6%
2	3819	12.7%
3	240	0.8%
4	76	0.3%
7	27	0.1%
6	23	0.1%
5	21	0.1%
1	4	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	4085	13.6%
-1	5938	19.8%
0	15764	52.5%
1	4	< 0.1%
2	3819	12.7%

Value	Count	Frequency (%)
8	3	< 0.1%
7	27	0.1%
6	23	0.1%
5	21	0.1%
4	76	0.3%

PAY_4
Real number (ℝ)

ZEROS

Distinct	11
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.2206666667
Minimum	-2
Maximum	8
Zeros	16455
Zeros (%)	54.9%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.169138622
Coefficient of variation (CV)	-5.29821128
Kurtosis	3.496983496
Mean	-0.2206666667
Median Absolute Deviation (MAD)	0
Skewness	0.9996294133
Sum	-6620
Variance	1.366885118
Monotocity	Not monotonic

Histogram with fixed size bins (bins=11)

Value	Count	Frequency (%)
0	16455	54.9%
-1	5687	19.0%
-2	4348	14.5%
2	3159	10.5%
3	180	0.6%
4	69	0.2%
7	58	0.2%
5	35	0.1%
6	5	< 0.1%
8	2	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	4348	14.5%
-1	5687	19.0%
0	16455	54.9%
1	2	< 0.1%
2	3159	10.5%

Value	Count	Frequency (%)
8	2	< 0.1%
7	58	0.2%
6	5	< 0.1%
5	35	0.1%
4	69	0.2%

PAY_5
Real number (ℝ)

ZEROS

Distinct	10
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.2662
Minimum	-2
Maximum	8
Zeros	16947
Zeros (%)	56.5%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.133187406
Coefficient of variation (CV)	-4.256902352
Kurtosis	3.989748144
Mean	-0.2662
Median Absolute Deviation (MAD)	0
Skewness	1.008197025
Sum	-7986
Variance	1.284113697
Monotocity	Not monotonic

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	16947	56.5%
-1	5539	18.5%
-2	4546	15.2%
2	2626	8.8%
3	178	0.6%
4	84	0.3%
7	58	0.2%
5	17	0.1%
6	4	< 0.1%
8	1	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	4546	15.2%
-1	5539	18.5%
0	16947	56.5%
2	2626	8.8%
3	178	0.6%

Value	Count	Frequency (%)
8	1	< 0.1%
7	58	0.2%
6	4	< 0.1%
5	17	0.1%
4	84	0.3%

PAY_6
Real number (ℝ)

ZEROS

Distinct	10
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.2911
Minimum	-2
Maximum	8
Zeros	16286
Zeros (%)	54.3%
Memory size	234.4 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.149987626
Coefficient of variation (CV)	-3.950489954
Kurtosis	3.42653413
Mean	-0.2911
Median Absolute Deviation (MAD)	0
Skewness	0.9480293916
Sum	-8733
Variance	1.322471539
Monotocity	Not monotonic

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	16286	54.3%
-1	5740	19.1%
-2	4895	16.3%
2	2766	9.2%
3	184	0.6%
4	49	0.2%
7	46	0.2%
6	19	0.1%
5	13	< 0.1%
8	2	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	4895	16.3%
-1	5740	19.1%
0	16286	54.3%
2	2766	9.2%
3	184	0.6%

Value	Count	Frequency (%)
8	2	< 0.1%
7	46	0.2%
6	19	0.1%
5	13	< 0.1%
4	49	0.2%

BILL_AMT1
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	22723
Distinct (%)	75.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	51223.3309
Minimum	-165580
Maximum	964511
Zeros	2008
Zeros (%)	6.7%
Memory size	234.4 KiB

Quantile statistics

Minimum	-165580
5-th percentile	0
Q1	3558.75
median	22381.5
Q3	67091
95-th percentile	201203.05
Maximum	964511
Range	1130091
Interquartile range (IQR)	63532.25

Descriptive statistics

Standard deviation	73635.86058
Coefficient of variation (CV)	1.437545339
Kurtosis	9.806289341
Mean	51223.3309
Median Absolute Deviation (MAD)	21800.5
Skewness	2.663861022
Sum	1536699927
Variance	5422239963
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2008	6.7%
390	244	0.8%
780	76	0.3%
326	72	0.2%
316	63	0.2%
2500	59	0.2%
396	49	0.2%
2400	39	0.1%
416	29	0.1%
500	25	0.1%
Other values (22713)	27336	91.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-165580	1	< 0.1%
-154973	1	< 0.1%
-15308	1	< 0.1%
-14386	1	< 0.1%
-11545	1	< 0.1%

Value	Count	Frequency (%)
964511	1	< 0.1%
746814	1	< 0.1%
653062	1	< 0.1%
630458	1	< 0.1%
626648	1	< 0.1%

BILL_AMT2
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	22346
Distinct (%)	74.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	49179.07517
Minimum	-69777
Maximum	983931
Zeros	2506
Zeros (%)	8.4%
Memory size	234.4 KiB

Quantile statistics

Minimum	-69777
5-th percentile	0
Q1	2984.75
median	21200
Q3	64006.25
95-th percentile	194792.2
Maximum	983931
Range	1053708
Interquartile range (IQR)	61021.5

Descriptive statistics

Standard deviation	71173.76878
Coefficient of variation (CV)	1.447236829
Kurtosis	10.30294592
Mean	49179.07517
Median Absolute Deviation (MAD)	20810
Skewness	2.705220853
Sum	1475372255
Variance	5065705363
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2506	8.4%
390	231	0.8%
326	75	0.2%
780	75	0.2%
316	72	0.2%
2500	51	0.2%
396	51	0.2%
2400	42	0.1%
-200	29	0.1%
416	28	0.1%
Other values (22336)	26840	89.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-69777	1	< 0.1%
-67526	1	< 0.1%
-33350	1	< 0.1%
-30000	1	< 0.1%
-26214	1	< 0.1%

Value	Count	Frequency (%)
983931	1	< 0.1%
743970	1	< 0.1%
671563	1	< 0.1%
646770	1	< 0.1%
624475	1	< 0.1%

BILL_AMT3
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	22026
Distinct (%)	73.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	47013.1548
Minimum	-157264
Maximum	1664089
Zeros	2870
Zeros (%)	9.6%
Memory size	234.4 KiB

Quantile statistics

Minimum	-157264
5-th percentile	0
Q1	2666.25
median	20088.5
Q3	60164.75
95-th percentile	187821.05
Maximum	1664089
Range	1821353
Interquartile range (IQR)	57498.5

Descriptive statistics

Standard deviation	69349.38743
Coefficient of variation (CV)	1.475106015
Kurtosis	19.78325514
Mean	47013.1548
Median Absolute Deviation (MAD)	19708.5
Skewness	3.087830046
Sum	1410394644
Variance	4809337537
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2870	9.6%
390	275	0.9%
780	74	0.2%
326	63	0.2%
316	62	0.2%
396	48	0.2%
2500	40	0.1%
2400	39	0.1%
416	29	0.1%
200	27	0.1%
Other values (22016)	26473	88.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-157264	1	< 0.1%
-61506	1	< 0.1%
-46127	1	< 0.1%
-34041	1	< 0.1%
-25443	1	< 0.1%

Value	Count	Frequency (%)
1664089	1	< 0.1%
855086	1	< 0.1%
693131	1	< 0.1%
689643	1	< 0.1%
689627	1	< 0.1%

BILL_AMT4
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	21548
Distinct (%)	71.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	43262.94897
Minimum	-170000
Maximum	891586
Zeros	3195
Zeros (%)	10.7%
Memory size	234.4 KiB

Quantile statistics

Minimum	-170000
5-th percentile	0
Q1	2326.75
median	19052
Q3	54506
95-th percentile	174333.35
Maximum	891586
Range	1061586
Interquartile range (IQR)	52179.25

Descriptive statistics

Standard deviation	64332.85613
Coefficient of variation (CV)	1.487019671
Kurtosis	11.30932483
Mean	43262.94897
Median Absolute Deviation (MAD)	18656
Skewness	2.821965291
Sum	1297888469
Variance	4138716378
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	3195	10.7%
390	246	0.8%
780	101	0.3%
316	68	0.2%
326	62	0.2%
396	44	0.1%
150	39	0.1%
2400	39	0.1%
2500	34	0.1%
1000	33	0.1%
Other values (21538)	26139	87.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-170000	1	< 0.1%
-81334	1	< 0.1%
-65167	1	< 0.1%
-50616	1	< 0.1%
-46627	1	< 0.1%

Value	Count	Frequency (%)
891586	1	< 0.1%
706864	1	< 0.1%
628699	1	< 0.1%
616836	1	< 0.1%
572805	1	< 0.1%

BILL_AMT5
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	21010
Distinct (%)	70.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	40311.40097
Minimum	-81334
Maximum	927171
Zeros	3506
Zeros (%)	11.7%
Memory size	234.4 KiB

Quantile statistics

Minimum	-81334
5-th percentile	0
Q1	1763
median	18104.5
Q3	50190.5
95-th percentile	165794.3
Maximum	927171
Range	1008505
Interquartile range (IQR)	48427.5

Descriptive statistics

Standard deviation	60797.15577
Coefficient of variation (CV)	1.508187617
Kurtosis	12.30588129
Mean	40311.40097
Median Absolute Deviation (MAD)	17688.5
Skewness	2.876379867
Sum	1209342029
Variance	3696294150
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	3506	11.7%
390	235	0.8%
780	94	0.3%
316	79	0.3%
326	62	0.2%
150	58	0.2%
396	47	0.2%
2400	39	0.1%
2500	37	0.1%
416	36	0.1%
Other values (21000)	25807	86.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-81334	1	< 0.1%
-61372	1	< 0.1%
-53007	1	< 0.1%
-46627	1	< 0.1%
-37594	1	< 0.1%

Value	Count	Frequency (%)
927171	1	< 0.1%
823540	1	< 0.1%
587067	1	< 0.1%
551702	1	< 0.1%
547880	1	< 0.1%

BILL_AMT6
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	20604
Distinct (%)	68.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	38871.7604
Minimum	-339603
Maximum	961664
Zeros	4020
Zeros (%)	13.4%
Memory size	234.4 KiB

Quantile statistics

Minimum	-339603
5-th percentile	0
Q1	1256
median	17071
Q3	49198.25
95-th percentile	161912
Maximum	961664
Range	1301267
Interquartile range (IQR)	47942.25

Descriptive statistics

Standard deviation	59554.10754
Coefficient of variation (CV)	1.53206613
Kurtosis	12.27070529
Mean	38871.7604
Median Absolute Deviation (MAD)	16755
Skewness	2.846644576
Sum	1166152812
Variance	3546691724
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	4020	13.4%
390	207	0.7%
780	86	0.3%
150	78	0.3%
316	77	0.3%
326	56	0.2%
396	45	0.1%
416	36	0.1%
-18	33	0.1%
2400	32	0.1%
Other values (20594)	25330	84.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-339603	1	< 0.1%
-209051	1	< 0.1%
-150953	1	< 0.1%
-94625	1	< 0.1%
-73895	1	< 0.1%

Value	Count	Frequency (%)
961664	1	< 0.1%
699944	1	< 0.1%
568638	1	< 0.1%
527711	1	< 0.1%
527566	1	< 0.1%

PAY_AMT1
Real number (ℝ_≥0)

ZEROS

Distinct	7943
Distinct (%)	26.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5663.5805
Minimum	0
Maximum	873552
Zeros	5249
Zeros (%)	17.5%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	1000
median	2100
Q3	5006
95-th percentile	18428.2
Maximum	873552
Range	873552
Interquartile range (IQR)	4006

Descriptive statistics

Standard deviation	16563.28035
Coefficient of variation (CV)	2.924524575
Kurtosis	415.2547427
Mean	5663.5805
Median Absolute Deviation (MAD)	1932
Skewness	14.66836433
Sum	169907415
Variance	274342256.1
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	5249	17.5%
2000	1363	4.5%
3000	891	3.0%
5000	698	2.3%
1500	507	1.7%
4000	426	1.4%
10000	401	1.3%
1000	365	1.2%
2500	298	1.0%
6000	294	1.0%
Other values (7933)	19508	65.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	5249	17.5%
1	9	< 0.1%
2	14	< 0.1%
3	15	0.1%
4	18	0.1%

Value	Count	Frequency (%)
873552	1	< 0.1%
505000	1	< 0.1%
493358	1	< 0.1%
423903	1	< 0.1%
405016	1	< 0.1%

PAY_AMT2
Real number (ℝ_≥0)

SKEWED
ZEROS

Distinct	7899
Distinct (%)	26.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5921.1635
Minimum	0
Maximum	1684259
Zeros	5396
Zeros (%)	18.0%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	833
median	2009
Q3	5000
95-th percentile	19004.35
Maximum	1684259
Range	1684259
Interquartile range (IQR)	4167

Descriptive statistics

Standard deviation	23040.8704
Coefficient of variation (CV)	3.891274139
Kurtosis	1641.631911
Mean	5921.1635
Median Absolute Deviation (MAD)	1991
Skewness	30.45381745
Sum	177634905
Variance	530881708.9
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	5396	18.0%
2000	1290	4.3%
3000	857	2.9%
5000	717	2.4%
1000	594	2.0%
1500	521	1.7%
4000	410	1.4%
10000	318	1.1%
6000	283	0.9%
2500	251	0.8%
Other values (7889)	19363	64.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	5396	18.0%
1	15	0.1%
2	20	0.1%
3	18	0.1%
4	11	< 0.1%

Value	Count	Frequency (%)
1684259	1	< 0.1%
1227082	1	< 0.1%
1215471	1	< 0.1%
1024516	1	< 0.1%
580464	1	< 0.1%

PAY_AMT3
Real number (ℝ_≥0)

ZEROS

Distinct	7518
Distinct (%)	25.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5225.6815
Minimum	0
Maximum	896040
Zeros	5968
Zeros (%)	19.9%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	390
median	1800
Q3	4505
95-th percentile	17589.4
Maximum	896040
Range	896040
Interquartile range (IQR)	4115

Descriptive statistics

Standard deviation	17606.96147
Coefficient of variation (CV)	3.36931393
Kurtosis	564.3112295
Mean	5225.6815
Median Absolute Deviation (MAD)	1795
Skewness	17.21663544
Sum	156770445
Variance	310005092.2
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	5968	19.9%
2000	1285	4.3%
1000	1103	3.7%
3000	870	2.9%
5000	721	2.4%
1500	490	1.6%
4000	381	1.3%
10000	312	1.0%
1200	243	0.8%
6000	241	0.8%
Other values (7508)	18386	61.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	5968	19.9%
1	13	< 0.1%
2	19	0.1%
3	14	< 0.1%
4	15	0.1%

Value	Count	Frequency (%)
896040	1	< 0.1%
889043	1	< 0.1%
508229	1	< 0.1%
417588	1	< 0.1%
400972	1	< 0.1%

PAY_AMT4
Real number (ℝ_≥0)

ZEROS

Distinct	6937
Distinct (%)	23.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4826.076867
Minimum	0
Maximum	621000
Zeros	6408
Zeros (%)	21.4%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	296
median	1500
Q3	4013.25
95-th percentile	16014.95
Maximum	621000
Range	621000
Interquartile range (IQR)	3717.25

Descriptive statistics

Standard deviation	15666.15974
Coefficient of variation (CV)	3.246147995
Kurtosis	277.3337677
Mean	4826.076867
Median Absolute Deviation (MAD)	1500
Skewness	12.90498482
Sum	144782306
Variance	245428561.1
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	6408	21.4%
1000	1394	4.6%
2000	1214	4.0%
3000	887	3.0%
5000	810	2.7%
1500	441	1.5%
4000	402	1.3%
10000	341	1.1%
2500	259	0.9%
500	258	0.9%
Other values (6927)	17586	58.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	6408	21.4%
1	22	0.1%
2	22	0.1%
3	13	< 0.1%
4	20	0.1%

Value	Count	Frequency (%)
621000	1	< 0.1%
528897	1	< 0.1%
497000	1	< 0.1%
432130	1	< 0.1%
400046	1	< 0.1%

PAY_AMT5
Real number (ℝ_≥0)

ZEROS

Distinct	6897
Distinct (%)	23.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4799.387633
Minimum	0
Maximum	426529
Zeros	6703
Zeros (%)	22.3%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	252.5
median	1500
Q3	4031.5
95-th percentile	16000
Maximum	426529
Range	426529
Interquartile range (IQR)	3779

Descriptive statistics

Standard deviation	15278.30568
Coefficient of variation (CV)	3.183386475
Kurtosis	180.0639402
Mean	4799.387633
Median Absolute Deviation (MAD)	1500
Skewness	11.12741705
Sum	143981629
Variance	233426624.4
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	6703	22.3%
1000	1340	4.5%
2000	1323	4.4%
3000	947	3.2%
5000	814	2.7%
1500	426	1.4%
4000	401	1.3%
10000	343	1.1%
500	250	0.8%
6000	247	0.8%
Other values (6887)	17206	57.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	6703	22.3%
1	21	0.1%
2	13	< 0.1%
3	13	< 0.1%
4	12	< 0.1%

Value	Count	Frequency (%)
426529	1	< 0.1%
417990	1	< 0.1%
388071	1	< 0.1%
379267	1	< 0.1%
332000	1	< 0.1%

PAY_AMT6
Real number (ℝ_≥0)

ZEROS

Distinct	6939
Distinct (%)	23.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5215.502567
Minimum	0
Maximum	528666
Zeros	7173
Zeros (%)	23.9%
Memory size	234.4 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	117.75
median	1500
Q3	4000
95-th percentile	17343.8
Maximum	528666
Range	528666
Interquartile range (IQR)	3882.25

Descriptive statistics

Standard deviation	17777.46578
Coefficient of variation (CV)	3.408581541
Kurtosis	167.1614296
Mean	5215.502567
Median Absolute Deviation (MAD)	1500
Skewness	10.64072733
Sum	156465077
Variance	316038289.4
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	7173	23.9%
1000	1299	4.3%
2000	1295	4.3%
3000	914	3.0%
5000	808	2.7%
1500	439	1.5%
4000	411	1.4%
10000	356	1.2%
500	247	0.8%
6000	220	0.7%
Other values (6929)	16838	56.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	7173	23.9%
1	20	0.1%
2	9	< 0.1%
3	14	< 0.1%
4	12	< 0.1%

Value	Count	Frequency (%)
528666	1	< 0.1%
527143	1	< 0.1%
443001	1	< 0.1%
422000	1	< 0.1%
403500	1	< 0.1%

default payment next month
Categorical

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	234.4 KiB

not default	23364
default	6636

Frequencies
Length

Value	Count	Frequency (%)
not default	23364	77.9%
default	6636	22.1%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	11
Median length	11
Mean length	10.1152
Min length	7

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Count
Matrix

First rows

	df_index	ID	LIMIT_BAL	SEX	EDUCATION	MARRIAGE	AGE	PAY_0	PAY_2	PAY_3	PAY_4	PAY_5	PAY_6	BILL_AMT1	BILL_AMT2	BILL_AMT3	BILL_AMT4	BILL_AMT5	BILL_AMT6	PAY_AMT1	PAY_AMT2	PAY_AMT3	PAY_AMT4	PAY_AMT5	PAY_AMT6	default payment next month
0	0	1	20000	female	university	1	24	2	2	-1	-1	-2	-2	3913	3102	689	0	0	0	0	689	0	0	0	0	default
1	1	2	120000	female	university	2	26	-1	2	0	0	0	2	2682	1725	2682	3272	3455	3261	0	1000	1000	1000	0	2000	default
2	2	3	90000	female	university	2	34	0	0	0	0	0	0	29239	14027	13559	14331	14948	15549	1518	1500	1000	1000	1000	5000	not default
3	3	4	50000	female	university	1	37	0	0	0	0	0	0	46990	48233	49291	28314	28959	29547	2000	2019	1200	1100	1069	1000	not default
4	4	5	50000	male	university	1	57	-1	0	-1	0	0	0	8617	5670	35835	20940	19146	19131	2000	36681	10000	9000	689	679	not default
5	5	6	50000	male	graduate school	2	37	0	0	0	0	0	0	64400	57069	57608	19394	19619	20024	2500	1815	657	1000	1000	800	not default
6	6	7	500000	male	graduate school	2	29	0	0	0	0	0	0	367965	412023	445007	542653	483003	473944	55000	40000	38000	20239	13750	13770	not default
7	7	8	100000	female	university	2	23	0	-1	-1	0	0	-1	11876	380	601	221	-159	567	380	601	0	581	1687	1542	not default
8	8	9	140000	female	high school	1	28	0	0	2	0	0	0	11285	14096	12108	12211	11793	3719	3329	0	432	1000	1000	1000	not default
9	9	10	20000	male	high school	2	35	-2	-2	-2	-2	-1	-1	0	0	0	0	13007	13912	0	0	0	13007	1122	0	not default

Last rows

	df_index	ID	LIMIT_BAL	SEX	EDUCATION	MARRIAGE	AGE	PAY_0	PAY_2	PAY_3	PAY_4	PAY_5	PAY_6	BILL_AMT1	BILL_AMT2	BILL_AMT3	BILL_AMT4	BILL_AMT5	BILL_AMT6	PAY_AMT1	PAY_AMT2	PAY_AMT3	PAY_AMT4	PAY_AMT5	PAY_AMT6	default payment next month
29990	30193	29991	140000	male	university	1	41	0	0	0	0	0	0	138325	137142	139110	138262	49675	46121	6000	7000	4228	1505	2000	2000	not default
29991	30194	29992	210000	male	university	1	34	3	2	2	2	2	2	2500	2500	2500	2500	2500	2500	0	0	0	0	0	0	default
29992	30195	29993	10000	male	high school	1	43	0	0	0	-2	-2	-2	8802	10400	0	0	0	0	2000	0	0	0	0	0	not default
29993	30196	29994	100000	male	graduate school	2	38	0	-1	-1	0	0	0	3042	1427	102996	70626	69473	55004	2000	111784	4000	3000	2000	2000	not default
29994	30197	29995	80000	male	university	2	34	2	2	2	2	2	2	72557	77708	79384	77519	82607	81158	7000	3500	0	7000	0	4000	default
29995	30198	29996	220000	male	high school	1	39	0	0	0	0	0	0	188948	192815	208365	88004	31237	15980	8500	20000	5003	3047	5000	1000	not default
29996	30199	29997	150000	male	high school	2	43	-1	-1	-1	-1	0	0	1683	1828	3502	8979	5190	0	1837	3526	8998	129	0	0	not default
29997	30200	29998	30000	male	university	2	37	4	3	2	-1	0	0	3565	3356	2758	20878	20582	19357	0	0	22000	4200	2000	3100	default
29998	30201	29999	80000	male	high school	1	41	1	-1	0	0	0	-1	-1645	78379	76304	52774	11855	48944	85900	3409	1178	1926	52964	1804	default
29999	30202	30000	50000	male	university	1	46	0	0	0	0	0	0	47929	48905	49764	36535	32428	15313	2078	1800	1430	1000	1000	1000	default

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows